AD-A149  543 


UNCLASSIFIED 


STATISTICAL  ANALVSIS  OF  AUTOREGRESSIVE  SPECTRAL  *r 

ESTIMATES  FOR  NOISE  CORRUPTED  AUTOREGRESSIVE  SERIES<U) 

NAVAL  OCEAN  SYSTEMS  CENTER  SAN  DIEGO  CA  D  F  GINGRAS 

OCT  84  NOSC/TR-979  F/G  12/1  NL 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUNIAU  OF  STANDARDS  I9b»  A 


AD-A149  543 


Technical  Report  979 

October  1984 

Interim  Report 

May  1983  —  January  1984 

STATISTICAL  ANALYSIS  OF 
AUTOREGRESSIVE  SPECTRAL 
ESTIMATES  FOR  NOISE  CORRUPTED 
AUTOREGRESSIVE  SERIES 

D.  F.  Gingras 

Prepared  for 

Office  of  Naval  Research 

Code  411 


Naval  Ocean  Systems  Center 


San  Diego,  California  92152 


Approved  for  public  release,  distribution  unlimited 


$ 


DTIC 

ELECTE| 

JAN  2  2  1985 


3 


85  01  14  0  Ig 


TR  979 


NAVAL  OCEAN  SYSTEMS  CENTER  SAN  DIEGO,  CA  92152 


AN  ACTIVITY  OF  THE  NAVAL  MATERIAL  COMMAND 
J.M.  PATTON,  CAPT,  USN  R.M.  HILLYER 

Commander  Technical  Director 

ADMINISTRATIVE  INFORMATION 

This  task  was  performed  for  the  Office  of  Naval  Research,  Code  411, 
Arlington,  VA,  22217,  under  program  element  61153N,  subproject  RR014110B  (NOSC 
ST73733).  This  work  was  carried  out  by  the  Naval  Ocean  Systems  Center,  Code 
733,  San  Diego,  CA  92152. 


Released  by 
D.  F.  Gingras,  Head 
Signal  Processing 
Technology  Branch 


Under  Authority  of 
R.  R.  Smith,  Head 
Signal  and  Information 
Processing  Division 


UNCLASSIFIED 


REPORT  DOCUMENTATION  PAGE 


UNCLASSIFIED 

2a  SECURITY  CLASS* ICATION  AUTHORITY 

I  2b  0€CIASSIF«CAT10N  DOWNGRADING  SCHEDULE 


3  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Approved  for  public  release; 
distribution  unlimited. 


4  PERFORMING  ORGANIZATION  REPORT  NUMB€R(Sl 

NOS C  TR  979 


6a  NAME  OF  PERFORMING  ORGANIZATION 

66  OfFICE  SYMBOL 
(if  appbcabfaj 

Naval  Ocean  Systems  Center 

Code  733 

6c  ADORESS  /City  Stata  and  ZIP  Codai 

San  Diego.  CA  92152 

6a  NAME  OF  FUNDING  SPONSORING  ORGANIZATION 

86  OFFICE  SYMBOL 

Id  afipbcabbl 

Office  of  Naval  Research 

Code  411 

0c  ADDRESS  (Cay  Stata  and  ZIP  Codai 

Arlington,  VA  22217 

6  MONITORING  ORGANIZATION  REPORT  NUMBERlSl 


7a  NAME  OF  MONITORING  ORGANIZATION 


9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


PROJECT  NO 

TASK  NO 

WORK  UNIT  NO 

RR014110B 

ST73733 

i 1  Tf7  lE  hncHjdm  Security  Classification / 

STATISTICAL  ANALYSIS  OF  AUTOREGRESSIVE  SPECTRAL  ESTIMATES  FOR  NOISE  CORRUPTED  AUTOREGRESSIVE  SERIES 


12  PERSONAL  AUTMOR(S) 

D.  F .  Gingras 


1  3 a  TYPE  Of  REPORT 

Interim 


136  TIME  COVERED 

from  May  83  to  Jan  84 


14  DATE  OF  REPORT  (Yaar.  Month.  OayJ 

October  1984 


16  PAGE  COUNT 
28 


1 7  COSAT1  COOES  I  1 B  SUBJECT  TERMS  (Contmua  on  ravarsa  if  nacaasary  and  idantify  by  bloc*  nombm) 

"ELD  |  group  |  sub  group  |  Spectral  Estimation 

Autoregressive  Series 
Asymptotic  Statistics 


1 9  ABSTRACT  iContinua  on  ravatta  if  nacassary  and  niantity  by  block  numbari 

I  stimation  of  the  spectral  density  function  for  a  gaussian  distributed  autoregressive  series  from  observations  of  a  noise  corrupted  version 
is  considered  when  the  order  of  the  autoregressive  series  is  assumed  to  be  known.  When  the  high-order  Yule-Walker  equation  estimates  of  the 
autoregressive  parameters  are  used  to  form  the  spectral  density  estimate,  it  is  shown  that  the  estimate  is  weakly  consistent  and  asymptotically 
normal  with  zero  mean  and  finite  variance.  A  closed  form  expression  for  the  asymptotic  variance  is  developed  and  the  expression  is  analyzed 
lor  the  lirst-order  AR  series  case. 


20  0'S TRIBUTION  AVAILABILITY  OE  ABSTRACT 

□  UNCLASSIFIED  UNLIMITED  □  SAME  AS  f 


22a  NAME  OF  RESPONSIBLE  INDIVIDUAL 

D.  I  .  Gingras 


OD  FORM  1473.  84  JAN 


21  ABSTRACT  SECURITY  CLASSIFICATK>*I 

[~1  one  users  UNCLASSIFIED 


22b  TELEPHONE  Itncludt  Arn  CoM / 

(619)  225-2381 


S3  APR  EDITION  MAY  SE  USED  UNTIL  EXHAUSTED 
ALL  OTHER  EDITIONS  ARE  OBSOLETE 


Accession  Number 
DN388506 


22c  OFFICE  SYMBOL 

733 


UNCLASSIFIED 


CONTENTS 


INTRODUCTION  .  .  .  page  1 
PRELIMINARIES  ...  1 
ASYMPTOTIC  PROPERTIES  ...  5 
EXAMPLE  ...  13 
REFERENCES  ...  19 
APPENDIX  ...  20 


ILLUSTRATIONS 


1.  Spectral  Estimate  Variance  vs.  Signal-to-Noise  Ratio.  AR  Parameter 

Equals  0.5,  A.  =  0  Radians  .  .  .  page  16 

2.  Spectral  Estimate  Variance  vs.  Signal-to-Noise  Ratio.  AR  Parameter 

Equals  0.1  and  0.8,  K  =  0  Radians  ...  18 

3.  Spectral  Estimate  Variance  vs.  Frequency  (radians).  Signal-to-Noise 

Ratio  equals  0  dB;  AR  Parameter  Equals  0.1,  0.5,  and  0.8  .  .  .  18 


I.  INTRODUCTION 


This  report  considers  the  problem  of  estimating  the  spectral  density  of  a 
discrete-time  autoregressive  (AR)  series  from  observations  of  a  noise 
corrupted  version.  The  spectral  density  estimate  is  based  on  the  high-order 
Yule-Walker  equation  estimates  of  the  AR  parameters.  Under  the  assumption 
that  the  order  of  the  autoregressive  series  is  known,  the  limiting 
distribution  of  the  spectral  density  estimate  is  normal  with  mean  zero  and 
finite  variance.  The  mean  and  variance  of  the  limiting  distribution,  for  the 
noise  corrupted  case,  have  not  previously  been  evaluated. 

The  problem  of  AR  parameter  estimation  for  the  noise  corrupted  case  was 
previously  considered  by  Walker  (reference  1),  Pagano  (reference  2),  and  Lee 
(reference  3).  Walker  was  the  first  to  consider  this  problem;  he  evaluated 
the  asymptotic  efficiency  and  variance  for  the  parameter  estimates  of  a  first 
order  series.  Pagano  proved  that  an  equivalent  model  for  an  autoregressive 
series  plus  noise  is  an  autoregressive-moving-average  (ARMA)  model.  Through 
the  use  of  nonlinear  regression  methods,  he  developed  strongly  consistent, 
efficient  parameter  estimates.  Lee  recently  examined  the  multivariate  noise 
corrupted  case  and  proved  that  the  multivariate  parameter  estimates  are 
strongly  consistent  and  asymptotically  normal. 

The  organization  of  this  paper  is  as  follows:  In  Section  II,  the  form  of 
the  spectral  density  and  the  AR  parameter  estimator  for  the  noise  corrupted 
case  is  established.  In  Section  III  proof  is  offered  that  the  limiting 
distribution  of  the  AR  spectral  density  estimate  is  normal  with  mean  zero  and 
the  asymptotic  variance  expression  is  evaluated.  In  Section  IV,  the  variance 
expression  for  the  first-order  Markov  series  (as  an  example)  is  evaluated. 

II.  PRELIMINARIES 

Let  {Yn}*__oa  be  a  discrete  parameter  time  series  satisfying  the  following 
assumption: 

Assumption  A:  The  series  {Yn}  consists  of  the  sum  of  an  autoregressive 

series  {X  }  of  known  order  p  and  a  noise  series  {W  }.  The  AR  series  {X  }  is 
n  n  n 

generated  (or  modeled)  by 


1 


(i)  {e n }  is  stationary  independent  identically  distributed  N(0,crz) 

2 

(ii)  {Wn}  is  stationary  independent  identically  distributed  N(0,ow) 

(iii)  and  {Wn}  are  uncorrelated 

The  parameter  set  { a j ) is  referred  to  as  the  AR  parameter  set. 

Assumption  B:  The  AR  parameters  are  constrained  such  that  the  zeros  of 
the  polynomial 


AP(z)  =  1  -  ajzJ 
j=l 


(2) 


lie  outside  of  the  unit  circle  on  the  complex  z-plane. 

Under  Assumption  B  the  AR  series  is  stationary.  It  was  assumed  that  the 
noise  is  wide-sense  stationary;  thus,  the  spectral  density  function  for  the 
noise  corrupted  series  Y  can  be  written  as 


=  2"  *  2nA"(.,x)Ap(e'U) 


(3) 


Walker  (reference  1)  and  Pagano  (reference  2)  showed  that  the  AR  plus 
noise  series  can  be  expressed  as  an  ARMA  series.  We  express  the  noise 
corrupted  series  Y  by 


Yn  "  alYn-l 


a  Y  =  e  +  w  -  a,w„  , 
p  n-p  n  n  1  n-1 


•  •  -  aw 

p  n-p 


(4) 


Let  the  covariance  sequence  of  the  series  Y  be  {r.},  where  r.  =  E[Y  Y  _.]. 

k  k  n  n  K 

Multiplying  (4)  through  by  Yn_k  and  taking  expectations  term  by  term  we  obtain 
the  Yule-Walker  (Y-W)  equations: 


(k  =  0) 


(5) 


ro  "  alrl  ‘ 
rk  '  alrk-l 
rk  "  alrk-l 


2  .  2 
a  r  =  o  +  o  , 
p  p  e  w 


■  a  r,  =  -a.  o 
p  k-p  k  w 


-  a  r.  =0 
P  k-p 


(1  <  k  <  p)  (6) 

(p+1  <  k  <  2p)  .  (7) 


The  set  of  p  equations  of  (7)  are  often  referred  to  as  the  high-order 
Yule-Walker  equations.  We  express  this  set  of  equations  in  matrix  form  as 


(8) 


where  the  (p  *  p)  covariance  matrix  fp  is  defined  by 


Lr2p-1  r2p-2  V 

and  the  (p  *  1)  vectors  a  and  Rp+1  are  defined  by 
aT  =  [a1,  a2,  • • • ,  ap] 

-p+1  =  [rp+l’  rp+2’  ***’  r2p]  • 


(9) 


Given  a  finite  set  of  observations  of  the  noise  corrupted  series  Y,  that 
N 

is  {Y  }  ,  N  >  2p,  we  estimate  the  covariance  sequence  {r. }  using 

n  n- 1  k 

!  N-|k| 

E  YnYn+| k|  'k*  ^  N'1 

n=l  (10) 

0  | k|  >  N-l 

When  the  covariances  r^  of  the  matrix  £p  and  the  vector  Rp+^  are  replaced  by 
their  corresponding  estimates  of  (10),  the  estimated  matrix  and  vector  will  be 
denoted  by  Fp  and  Rp+^.  respectively.  The  high-order  Y-W  equations  (8)  can  be 
expressed  in  terms  of  the  estimated  covariances  as 


3 


The  solution  of  (11)  in  terms  of  a  provides  the  high-order  Y-W  equation 
estimate  of  the  AR  parameters. 

In  order  to  estimate  the  AR  spectral  density  we  require  estimates  of  the 

2 

AR  parameters  such  as  those  formed  by  (11)  and  an  estimate  of  a£.  In  the 

noise  free  case,  given  estimates  of  the  covariances  {r^}  and  AR  parameters 

{a.}P  ,  (5)  can  be  used  to  estimate  a2.  For  the  noise  corrupted  case  (5)  will 
J  J  1  2  7  ^ 

provide  an  estimate  of  a  +  a  ,  thus,  one  of  the  equations  of  (6)  must  also  be 

r\  £  W 

used  to  estimate  o^.  Using  this  approach,  with  covariance  estimates  of  (10) 
and  estimates  of  the  AR  parameters  of  (11)  we  have 


~2 

a  -  - 
e 


E  Vj  '  <1/SP>  E  vj 

j=0  j=o 


where  a  =  -1  and  a  4  0. 
o  p 

In  the  subsequent  development  of  asymptotic  statistical  properties  for 
the  parameter  and  spectral  density  estimates,  we  make  use  of  the  following 
vectors  and  matrices: 

-T  ~  trl’  r2’  ***  •  r2p^ 


-  p  [rp’  rp+l’ 


’  r2p-l] 


-nm  -  {uk, j>  k  =  n*  n  +  1,  2p 


j  =  m,  m  +  1,  •••,  2p 


A  J(k+j)A  +  i(k- j)A 


4 


4 


III.  ASYMPTOTIC  PROPERTIES 

A.  AR  Parameter  Estimate  Statistics 

Define  the  AR  parameter  vector  0^  by 

0T  £  [o2,  ar  •••,  ap]  .  (13) 

Our  present  goal  is  to  establish  the  asymptotic  distribution  for  estimates  of 
the  AR  parameter  vector.  First,  we  present  the  asymptotic  distribution  of  the 
covariance  estimates  of  (10)  as  established  by  Brillinger  (reference  4). 

Theorem  1:  For  the  AR  plus  noise  series  Y,  under  Assumptions  A  and  B, 
the  elements  of  the  covariance  vector 

N5  (R  -  R) 

are  asymptotically  jointly  multivariate  normal  with  mean  zero  and  covariance 

n 

lim  E  {N5  (R  -  R),  N1*  (R  -  R)T}  =  In  f  U,,<|>2(A)dA  .  (14) 

N-*»  J  11 

-71 

The  following  lemma  establishes  the  existence  of  a  random  vector  Z  that 
is  equivalent  in  distribution  to  the  high  order  Y-W  AR  parameter  estimate 
vector  (a  -  a).  In  preparation  for  this  lemma  we  define  the  matrix  D  by 


Lemma  1:  For  the  AR  plus  noise  series  Y  there  exists  a  p  x  2p  matrix  D 
and  a  random  vector  Z  such  that 

N*5  (a  -  a)  ~  N1*  Z  =  N**  [r~ XD( R  -  R)] 


(15) 


where  ~  indicates  that  the  limit  distribution  as  N-*»  is  identical  for  both 
random  vectors. 


Proof:  Define  the  vector  V  by 


V  - 


LVPJ 


^  n5  (r_1  -  r_1)(R  ..  -  r  a)  . 
-p  -p  -p+i  -p- 


Since  £  is  positive  definite  and  £  - ►£  ,  element  by  element,  it  follows 

p  P  N-x»  p 


that  r"1 
“P 


N^oo 

P 


T  ,  and 

-p 


v  . 
J 


N-K» 


j  =  1,  2, 


(16) 


Let  (Q,  P)  be  the  underlying  probability  space.  For  arbitrary  e  >  0  and 
N  >  p  let 


Ae,N  =  {u)  C  Q:  1  v j  1  <  e’  J  =  1.  2,  ••• ,  p} 


then  for  all  w  C  A  since  Iv.l  <  e,  we  can  write 

e,N’  j' 


>1 


V  (Vl  '  = 


It  follows  that 


V  =  ^  [(j  -  a)  -  r'1  <Rp+1  -  rpa)3  . 

for  all  iu  C  A  N.  By  (16)  we  have  that  for  every  ae[0,l]  there  exists  a  N* 

£  ,  IN  £  9  Of 

such  that 


Since  the  selection  of  e  and  a  is  arbitrary  we  can  conclude  that 


-  a)  -  «V'(Vl  -  rpa)  -—>0 


By  the  definition  of  the  matrix  D  and  by  the  high-order  Y-W  equations  (8)  and 
(11)  we  can  write 


2(R  -  5)  =  Bp+1  '  V 


and  the  desired  result  follows  directly. 


□ 


2 

We  previously  established  an  estimator  for  the  variance  o£,  see  (12).  In 

the  following  lemma  we  establish  the  existence  an  equivalent  (in 

~2 

distribution)  random  variable  from  which  the  asymptotic  distribution  of  can 
be  evaluated.  In  preparation  for  the  lemma  we  define  the  vectors: 


[r, 


O’  1’ 


r2p-l] 


Vr 


Lemma  2:  For  the  AR  plus  noise  series  Y  there  exists  a  random  variable  4 
such  that 


a\)  ~  N*5  4  =  N^HCRq  -  RQ] 


(17) 


where 


H  =  -{('I,  aT.  0]  +  (l/ap)  [aT,  -1,  0]} 


7 


Proof:  By  (5),  (6),  and  (12)  we  write 


N^cr2  -  ct2)  =  N*5  -  y  a.r.+  y  a.r.  -  (1/a  )  y  a.r  . 

£  £  ^  J  J  ^  J  J  P  ^  J  P’J 

j=0  j=0  j=0 

P 

+  (1/a  )  y  a.r  .  \  . 

p  j  p-] 

j=0  1 

By  Gersch  (reference  5)  we  have  that  the  high-order  Y-W  equation  AR  parameter 
estimates  converge  in  probability  to  the  true  parameters  as  N~*».  Thus,  we  can 
wr  i  te 


N^(d2  -  a2)  ~  N*5 

£  £ 


y  r.(a.  -  a.)  -  y  a.(r.  -  r.) 

J  J  J  ^  J  J  J 
j=0  j=0 


-  (i/y  E  Vj  -  y  -  <i/aP>  E  yvi  ■  v/  •  <18) 

j=0  j=0  J 

Also,  by  the  convergence  in  probability  result  of  Gersch  (reference  5)  we  have 
,  P 

that  N^(Sj  '  aj)  - ►O  as  (j  =  1,  2,  — ,  p);  thus,  the  first  and  third 

terms  on  the  right-hand  side  of  (18)  converge  to  zero  and  the  desired  result 
follows  directly.  □ 

Theorem  2:  Under  Assumptions  A  and  B  the  AR  parameter  estimates  converge 
in  distribution  to  a  zero  mean  normal  random  vector,  that  is 


™  (0  -  0)  - -N  (0,  Z) 


where 


I  ^  lim  N  E 


«1  b  *T] 


8 


Proof:  This  result  follows  directly  from  the  results  of  Lemmas  1  and  2 


1 


and  Theorem  1. 


D 


We  now  proceed  to  evaluate  the  terms  of  I.  Let 


I  = 


21  T 
V£  C 


C  ,  $ 


Ji 


2  2 
where  V  ,  C,  and  1  are  defined  by  (19).  For  V  we  have 

V2  =  lim  N  E[42]  =  lim  N  H  E  {[Rq  -  R„][R  -  Rq]T}HT 

N  x»  ' 


and  by  Theorem  1  we  have 


= 


=  2 n  j  H  yQ0  HT  <J>2(A)dA 


After  further  manipulation  (see  appendix)  we  get 


w2  -  4 
M  -  a 

w 


P  P 

2 


£  ♦  <2/V  £  Vp-j  ' 1  +  (1/V  £ 

j=o  j=o  j=o 


,  2  2 
+  o  o 
w  e 


2/  E  aj )+  3  +  (1V 

j=0 


l*r 0  +  <2/ap)rp  *  <1/ap>Zr 


.] 


We  also  have 


CT  =  lim  N  E[4ZT]  =  lim  N  H  E{[Rn  -  Rn][R  -  R]}  DT  (r1) 
N-*»  N-x®  u  u  \  P  / 


(20) 


- j  ■ 


and  by  Theorem  1  we  have 


T 

f  M  u01  dt  (fp1)  ^(\)d\  . 


After  further  manipulation  (see  appendix)  we  get 


(21) 


where  the  vector  P  is  defined  by 


{PT}  £  -a2r 

J  1  £  p+1 


a4  y  a  .a  +1  . 

w  Lu  j  p+1  -  j 

j=l 


p-1 


(1/a  )  a4  a  .a .  , 
p  w  Zj  ]  j+1 

j=0 


d/a  )  a£r1 


1  =  1,  2,  •••,  p  . 


By  (15)  we  get 


$  =  lim  N  E[Z  ZT]  =  lim  N  r^O  E{[R  -  R][R  -  R]T}DT  (V'1) 
N-*»  N-x»  ™  \  P  / 


and  by  Theorem  1  we  can  write 
n 


T 

<3?  =  2n  J  Tp1  D  Un  DT  (fp1)  <f>y(A)dA  . 


-n 


After  further  manipulation  (see  appendix)  we  get 


$>  = 


„  2  r-l 

+  a  r 

w  -p 


i°l  i  *  °l  a^ip1) 


(22) 


where  |  is  the  p  *  p  identity  matrix  and  Q  is  given  by 


10 


P  2  p_1 

I  a  Z  a  a  •  • 
m=0  m  m=0  m  m+1 


p-1 

I 

m=0 


a  a  i 
m  m+1 


1 

m=0 


Z  a  a  ,  , 

m=0  ""•♦(P'D 


P 

z 

m=0 


a 


2 

m 


B.  AR  Spectral  Density  Estimates  Statistics 


We  now  proceed  to  evaluate  the  limiting  distribution  of  the  spectral 
density  estimate  for  the  AR  series  X  formed  from  observations  of  the  noise 
corrupted  series  Y.  From  (3)  we  see  that  the  AR  spectral  density  estimate 
can  be  written  in  terms  of  the  parameter  estimate  vector  0  as 


$X(A, 


27rAP(elA)AP(e‘lA) 


(23) 


p  *J  ^ 

where  the  estimate  A  (e  )  is  formed  by  substituting  the  AR  parameter 

i  A.  2 

estimates  of  (11)  into  (2)  and  evaluating  at  z  =  e  and  o£  is  estimated  using 
(12).  We  now  state  and  prove  the  main  result  of  the  document. 


Theorem  3:  Under  Assumptions  A  and  B  the  AR  spectral  density  estimate 
$X(A,  6)  converges  in  distribution  to  a  zero  mean  normal  random  variable,  that 
i  s 


nJ5[$x(a,  6)  -  <>X(A,  0)J 


(£> 

- -  N(0,  eT(A)  Z  fi(M) 

N-*» 


(24) 


where  £(\)  is  a  gradient  vector  given  by 


£T(A)  = 


3<1>X(A,  0)  8<|>X(A,  0) 


do 


9a, 


a*x(A,  e) 
8ap 


11 


Proof:  By  Theorem  2  we  have  that 


N^e  -  0)  - -  N  ,  (0,  I). 

N-x»  P 

Since  the  function  <>x ( A ,  0)  is  totally  differentiable  with  respect  to  the 
vector  e  the  desired  result  follows  directly  by  a  convergence  theorem  of  Rao 
(reference  6).  □ 

By  the  result  of  Theorem  3  we  see  that  from  observations  of  the  noise 
corrupted  AR  series  Y  ,  through  the  use  of  the  high-order  (Y-W)  equations,  we 
can  form  a  weakly  consistent  spectral  estimate  for  the  nonnoise  corrupted 

series  X  ;  the  resulting  spectral  estimate  is  asymptotically  normal  with 

n  T  T 

limiting  variance  (1/N)  £  (A)  I  £(A).  We  now  express  £  (A)  I  £(A)  in  terms  of 

previously  defined  terms. 

Let  the  gradient  vector  £^(A)  be  written  as 
£T(A)  ^  C b ( A ) ,  BT(A) ] 


where 


A  3<|)y(A,  0) 
b(A)  ^  X  7  ~ 
do 


2nAp(eU)Ap(e‘U) 


(25) 


and 


BT(A)  £ 


a<t>x(A,  0)  a$x(A,  0) 


a<l>x(A,  0) 


3a, 


3a, 


da 


<J>X  ( A , 


We  previously  defined  the  matrix  I  by 


(26) 


I 


thus,  we  can  write 

eT(\)  I  g(A)  =  b2(A)V2  +  b(A)  BT(A)  C  +  b(A)  CT  B(A)  +  BT(A)  $  B(A)  .  (28) 

In  the  next  section,  (28)  will  be  evaluated  for  the  first  order  AR 
series. 

IV.  EXAMPLE 

First  Order  Autoregressive  (Markov)  Series 


The  first  order  AR  series  (p  =  1)  is  given  by 


X  =  a  X  ,  +  e 
n  n-1  n 


where  the  parameter  a  must  satisfy  the  condition  -l<a<l,  i.e. ,  Assumption  B, 
for  the  series  to  be  stationary.  For  this  first  order  example  the  covariance 
sequence  is  expressed  by 

a2  a'kl 

r  =  - —  k  =  0,  1,  (30) 

(1  -  a  j 

and  the  spectral  density  is 

2 

a 

<MA,  0)  =  - p - 5 - 

2n[a^  -  2a  cos  A  +  1] 

T  2 

where  6  =  (o  ,  a). 

Given  observations  of  the  noise  corrupted  version  of  the  AR  series,  the 

variance  of  the  spectral  density  estimate,  as  given  by  (28)  can  be  evaluated 

2  2 

in  terms  of  the  parameter,  a,  and  the  variances  a  and  a  .  For  the  first 

£  W 

order  case,  p  =  1,  we  have 


i 
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Substituting  (34),  (35),  and  (36)  into  (31)  yields 


lim  N  var{<j>y(A,  0)} 
N-*» 


2  (  -a4  +  4a2  +  1 


(2n)2[a2  -  2a  cos  A  +  l]2  6  (  a2(l  -  a2) 


2a  (cos  A  -  a)  ,,  ,2 

_ £ _  ~  a 

(2n)2[a2  -  2a  cos  A  +  l]3  a 


1  [•: 


2  +  (o4/a2)(l  -  a2) 

e  v  w  e  ' 


4  2 

4a  (cos  A  -  a)  ,, 

_ £ _  U  ~  c 

(2n)2[a2  -  2a  cos  A  +  l]4  a2 


_i+M 


,/af  (1  -  a2) 


°2X 


We  see  that  the  limiting  variance  expression  is  composed  of  three  major 

terms:  the  first  term  represents  the  contribution  due  to  variation  in 

2 

estimating  a  ,  the  second  term  represents  the  contribution  due  to  the 
•  2 

cross-covariance  between  ag  and  a,  and  the  third  term  represents  the 
contribution  due  to  variation  in  estimating  the  parameter  a. 


Even  for  the  first-order  AR  series  we  see  from  (37)  that  the  limiting 

variance  expression  is  a  complicated  function  of  the  parameters.  To  provide 

some  insight  into  the  relationship  between  the  variance  and  the  parameters  a, 
2  2 

o£,  aw  and  A,  (37)  was  evaluated  for  a  few  parameter  values. 

Figure  1  shows  the  spectral  density  estimate  variance  plotted  as  a 
function  of  signal-to-noise  ratio  (SNR).  The  AR  series  used  is  given  by 


4 
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-22  -16  -10  -4  2  8  14  20 

SNR  (68) 

Figure  1 .  Spectral  estimate  variance  vs  signal-to-noise  ratio.  AR  parameter 
equals  0.5,  X=  0  radians. 
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(38) 


with  a  =  0.5;  low  pass  spectral  density.  The  variance  (37)  was  evaluated  for 
2  2 

\  =  0,  a  =1  and  a  set  to  achieve  the  indicated  SNRs.  We  see  that  the 
e  w 

variance  decreases  monotonical ly  with  increasing  SNR  to  a  value  of  14  dB  at  a 
SNR  of  8  dB.  Also  plotted  is  the  variance  obtained  by  Akaike  (reference  7) 
for  the  first-order  AR  series  without  noise  as  indicated  by  the  horizontal 
line  at  about  7  dB.  Note  that  the  AR  plus  noise  case  variance,  at  high  SNR, 
does  not  asymptotically  approach  the  no  noise  variance.  This  is  the  case 
because  the  high-order  Y-W  equation  estimates  of  the  AR  parameter  used  for  the 
AR  plus  noise  case  produces  a  larger  parameter  estimate  variance  than  that  of 
the  conventional  Y-W  equation  estimate.  That  is,  for  the  first-order  AR 
series  with  a  =  0.5  we  have  from  Akaike  (reference  7)  that 

lim  N  var(a)  =  (1  -  a2)  =  0.75 
N-x» 


and  for  the  AR  plus  noise  case  we  have  from  the  third  term  on  the  right  side 

of  (37)  with  a2  =  0  that 
w 

lim  N  var(a)  =  (1  -  a2)/a2  =  3.0 
N-*<» 


In  figure  2  the  estimated  variance  is  plotted  as  a  function  of  SNR  for 
A.  =  0  for  two  values  of  the  AR  parameter,  a  =  0.1  and  a  =0.8.  We  see  the  same 
monotonic  decrease  with  increasing  SNR  for  both  cases  as  in  figure  1.  The 
asymptotic  limit  for  the  a  =  0.8  case  is  about  15  dB  greater  than  that  for 
a  =  0.1.  Thus  indicating  that  as  the  spectral  density  bandwidth  decreases  the 
spectral  estimate  variance  increases.  In  figure  3  we  have  spectral  estimate 
variance  plotted  as  a  function  of  frequency  for  three  values  of  the  AR 
parameter  a  =  0.1,  a  =  0.5,  and  a  =  0.8.  We  see  that  for  the  two  narrower 
bandwidth  cases,  a  =  0.5  and  0.8,  the  variance  decreases  monotonical ly  with 
increasing  frequency  but  for  the  wider  bandwidth  case  a  =  0.1  there  is  little 
variation  with  frequency  over  the  range  evaluated. 
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SNR  (dB) 


2  8  14  20 

Figure  2.  Spectral  estimate  variance  vs  signal-to-noise  ratio. 

AR  parameter  equals  0.1  and  0.8,  X  =  0  radians. 


FREQUENCY  (radiant) 


Figure  3.  Spectral  estimate  variance  vs  frequency  (radians). 
Signal-to-noise  ratio  equals  0  dB;  AR  parameter  equals  0.1, 0.5,  and  0.8. 
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VI.  APPENDIX 


A.  Evaluation  of  <$>: 


From  Section  III  we  have 

71 

rp  $  rj  =  2tt  f  o  Un  oT  *y(\)d\. 


(A-l) 


We  first  evaluate  the  n,  mth  element  of  D  and  have 


2p  2p 

{D  IL ,  DT}  =  d  .  u.  .  d„.  n  =  1,  •••,  p;  m 

i - 11  -  1  nm  Z— <  nk  kj  mj  ’ 

k=l  j=l 


=  1,  • *  * ,  P- 


By  the  definition  of  the  matrix  we  have 

<ou  °T>™  -  i  £  dnk .y, 

k=l  j=l 


and  by  the  definition  of  the  matrix  D  we  get 


[0  uu  DTJnm  =  AP(e-,x)  eUptn)x{Ap(e'U)  ei(ptm)A  -  Ap(eU)e‘i(pt")X}.  (A-2) 
Substituting  (A-2)  into  (A-l)  we  get 


tp$^  =  Ii  +  I2 


where 

n 

{Tl}nm  -  2n  f  A(e'U)  A(e"lX)  el(2p+n+m)X  ^(A)d\  (A-3) 

-7t 

and 


20 


I 


{T0}  =  2n 

7  ’  nm 


a,JK  J(n-m)A  .2 


l)  A(e  )  e 


♦5(X)dA 


(A-4) 


For  the  AR(p)  plus  noise  process  we  have  that 


!  0  2  2  4 

4  ^°wae  °c 

*  iiVViT1*)  *  [A>VVVU>]2 


(A-5) 


Substituting  the  expression  (A-5)  into  (A- 3 )  and  (A-4)  and  carrying  out  the 
integration  we  get 


^nm  =  0 


n  =  1,  *  *  * ,  p;  m  =  1,  • • • ,  p 


p- I n-m| 

4  V'  .  .  .22 


4  v~'  2  2  2 

{T„}  =  a  >  a.a.  .  .  +  a  a  6(n-m)  +  o  r 

l-2‘nm  w  j  j+|n-m|  w  e  v  '  e  n-n 

j=0 


Using  these  results  we  get 


<t>  =  a2  r'1  rn  (r'A  +  a2  r_1[a2  I  +  a2  gj/r'A 

-  e  -p  -0  l-p  J  w  -p  c  -  w  s  l -p  J 


(A-6) 


where  the  matrix  g  was  defined  in  Section  III. 


B.  Evaluation  of  C  : 


From  Section  III  we  have 


=  2n  j  H  UQ1  <(.2(\)dA 


(A-7) 


with  H  defined  in  Lemma  2.  Substituting  the  expression  for  H  into  (A-7)  we 
have 
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4»yCA)cIA 


-  2n(l/ap) 


/ 1 

fii 


■l,  o]  u01  d' 


-71 


♦yCMdA 


(A-8) 


+  I2 


We  first  examine  the  contribution  due  to  T, .  By  the  definition  of  the 

x.  h  ^  T  T 

matrices  UQ1  and  D  we  can  write  the  i  '  element  of  [-1,  a  ,  0]  UQ1  D  by 


{[-1,  a1 


23  Ufli  2T), 


P 

z 

k=0 


ak  e 


p+1 

(kA£ 

j=l 


p-j-1 


{e 


+  e‘ij'A} 


1  =  1,2,  —  ,p 


=  Ap(eU)Ap(e'U)  ei(p+1)A  +  Ap(eiX)Ap(eil VUp+1  )X 


(A-9) 


Thus,  by  (A-9)  and  (A-8)  we  have 


71 

rj),  =  -Zn  f  (P+0^  ,  AP(e*  V(eU)eH  (p+1  ^(MdA 

-n 


(A- 10) 


Evaluating  we  get 

7t 

{S1}]  =  -2tt  j  AP(elA)AP(e'lA)ei(p+1)A  ^(A)dX 

—  7t 


Using  ( A- 5 )  and  carrying  out  the  integration  yields 


{S2}1  =  -27t  J  Ap(elA)Ap(eiA)e"i(p+1)A  *2 


and  by  (A- 5)  we  get 


4>y(X)dA 


(52>i  =  -'w  Z  ajVi-j 
j=l 


(A- 12) 


For  J2  we  have  by  the  definition  of  UQ;.  and  D  that  the  1th  element  of 


(l/ap)  [aT,  “1,  0]  yQ1  can  be  expressed  as 

<1/ap>  !CIT,  -1,  0]  U01  DT},  =  £  ap.k  £  ap.j+1 

k=0  j=l 


=  Ap(e~lA)Ap(e~lA)ei(2p+1)A  +  Ap(e'iA)Ap(elA)e_1 1A 


71 

{l2  Ep}  !  =  -2n  J {Ap(e'iA)Ap(e"iA)ei(2p+1)A  +  Ap(e‘iA)Ap(eiA)e'iU}*2(\)dA 


-  ^3>1  +  ihh  ’ 


(A-13) 


Evaluating  S3  using  (A-5)  for  <|>2(A)  we  get 


{hh  =  0 


1  -  1,  2,  • • • ,  p 


(A- 14) 


and  evaluating  S.  we  get 


Define  the  1  element  of  the  vector  P  by 


«T,i  =  W  *  {i*  4}i 


then  using  (A-ll)  and  (A- 12)  in  (A- 10)  and  (A- 14)  and  (A-15)  in 

p  P"1 

{^1  =  *°eVl  •  E  ajVl-j  "  (1/ap)oi  E  ajaj+l  • 

j=i  j=o 


1  =  1,  2,  •••,  p. 


It  follows  that 


cT  ■  £Tfc1' 


2 

C.  Evaluation  of  V  : 

From  Section  III  we  have 

71 

V2  =  2ti  j  H  y0Q  HT  <J>2  (A)dA 

-71 

where  H  was  defined  in  Lemma  2.  Using  the  expression  for  H  in 

71 

V2  =  2n  J  [-1,  aT,  0]  U0Q[-1,  aT,  0]T  <t>2(\)d\ 

-7t 

71 

+  2 n  |  [-1,  aT,  0]  U00[aT,  -1,  0](l/ap)*2(A)dA 

-7t 

7t 

+  2tt  J  (l/ap)  [aT,  -1,  0]  U0()[-l,  aT,  0]T  « 
-n 

71 

+  271  f  (1/a  )2  (aT,  -1,  0]  U00C1T,  -1 


(A-13)  we  get 


(A- 16) 

(A- 16)  we  get 


2(Md\ 

,  0]<|>2(A)dA 


By  the  definition  of  the  matrix  U  „  we  have  for  T, 

J  oo  1 

Ti  =  2n  fii 

-7i  k=0  j=0 

n 

=  2n  J  [Ap(eiA)Ap(eU)  +  Ap(eiA)Ap(e"iA)]^(A)dA 

“7t 

2 

Using  (A-5)  for  <}>y(A)  and  performing  the  integrations  we  get 


For  T2  we  have 


P  P 


T2  -  *  /  n/V  I  X  ajVkte«^^  ♦ 

~n  j=0  K=0 


again  using  (A-5)  and  performing  the  integrations  we  get 


T2  = 


-cr^  +  (1/a  )o^r  +  (1/a  )ct^  Y''  a  .a 
w  pep  p  w  L~i  j  p-j 

j=0 


For  we  have 


h  =  Zn  f  (l/ap)  ^  X  VA  •’’V*  *  •'*> 

j=0  k=0 


P  P 


71 


=  2tt  J  (l/ap)[Ap(elA)Ap(e"lA)eipA  +  A(eiA)A(eiA)e"ipA]<^(A)dA 


Using  (A-5)  and  performing  the  integrations  we  get 


T3  =  _aw  +  (1/ap)aerp  +  (1/ap)CTw  2-  ajap-j  +  2CTw 

j=0 


For  we  have 


P  P 


t4  -  *  /  ("V2  E  E  VjVk +  •'1JX»5»>* 

-n  j=0  k=0 


<1 

=  2n  J  (l/ap)2[AP(e"iX)Ap(e‘iX)ei2pX  +  Ap(e"lX)AP(elX)]<|>2(MdA 


Using  (A-5)  and  performing  the  integrations  we  get 


(1/a  )2  <o4  V  a2  +  2a2  a2  +  a2(rn  -  a*) 
p'  )  w  j  w  £  £v  0  wy 

(  j=0 


Thus, 


v2  =  Z  aj +  (2/aP)  E  ajaP-j " 1  +  (i/aP)Z  Z  aj 


°W°£^2/  Z  ajj+  3  +  (1V 


+  a2 {a2  +  rrt  +  (2/a  >r  +  (1/aJ2  rn} 


